This is our March drop for Index Advisor service for N1QL after fixing some of the bugs that were found after our last refresh in Feb. We plan to keep improvising the Index Advisor service(What is it?). The fixes in Index Advisor that you would normally see in the next maintenance release will now be available to you every month.
Who is it for? and When should you use it?
http://index-advisor.couchbase.com/
At the cost of repeating myself,
This service will provide index recommendations to help DBAs, developers, and architects optimize query performance and meet the SLAs.
You will find this service useful if you:
- Want to avoid reading the index creation rules, understand them, and implement them to find the appropriate indexes for your query/queries/workload.
- Do not want to download the latest Couchbase 6.5 server yet.
- Are using an older Couchbase version(version 5.5,6.0) and need help creating the right indexes for your queries.
- Want to generate advice for indexes without creating a bucket or uploading the schema or data.
Whats new?
Even if you have the Couchbase Server 6.5 server downloaded and can use Index Advisor from Query Workbench, this index Advisor is a more recent version(without a great looking UI) but with the following defects fixed.
1.Index Advisor to support TTL (meta().expiration & meta.cas)
With this improvement, a query like
|
1 2 3 4 5 |
ADVISE SELECT META().id, META().expiration FROM `travel-sample` WHERE META().expiration = 0 ORDER BY META().id LIMIT 2; |
that used to give
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
<span style="color: #ff0000">{ "results": [ { "#operator": "Advise", "advice": { "#operator": "IndexAdvice", "adviseinfo": { "recommended_indexes": "No index recommendation at this time: no keyspace found." } }, "query": "SELECT name, META().id\nFROM `travel-sample`\nWHERE META().id > \"g\"\nLIMIT 2;" } ] }</span> |
now gives
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
<span style="color: #339966">{ "results": [ { "#operator": "Advise", "advice": { "#operator": "IndexAdvice", "adviseinfo": { "recommended_indexes": { "covering_indexes": [ { "index_statement": "CREATE INDEX adv_meta_self_expiration ON `travel-sample`(meta(self).`expiration`)", "keyspace_alias": "travel-sample" } ], "indexes": [ { "index_statement": "CREATE INDEX adv_meta_self_expiration ON `travel-sample`(meta(self).`expiration`)", "keyspace_alias": "travel-sample", "recommending_rule": "Index keys follow order of predicate types: 2. equality/null/missing." } ] } } }, "query": "SELECT META().id, META().expiration\nFROM `travel-sample`\nWHERE META().expiration = 0\nORDER BY META().id\nLIMIT 2;" } </span> <span style="color: #339966">] }</span> |
or
|
1 2 3 4 5 |
ADVISE SELECT name, META().cas FROM `travel-sample` WHERE type="hotel" ORDER BY META().cas DESC LIMIT 2; |
that used to give:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
<span style="color: #ff0000">{ "results": [ { "#operator": "Advise", "advice": { "#operator": "IndexAdvice", "adviseinfo": { "recommended_indexes": { "indexes": [ { "index_statement": "CREATE INDEX adv_type ON `travel-sample`(`type`)", "keyspace_alias": "travel-sample", "recommending_rule": "Index keys follow order of predicate types: 2. equality/null/missing." } ] } } }, "query": "SELECT name, META().cas FROM `travel-sample` WHERE type=\"hotel\" ORDER BY META().cas DESC LIMIT 2;" } ] }</span> |
now gives
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
<span style="color: #339966">{ "results": [ { "#operator": "Advise", "advice": { "#operator": "IndexAdvice", "adviseinfo": { "recommended_indexes": { "covering_indexes": [ { "index_property": "ORDER pushdown, LIMIT pushdown", "index_statement": "CREATE INDEX adv_type_meta_self_casDESC_name ON `travel-sample`(`type`,meta(self).`cas` DESC,`name`)", "keyspace_alias": "travel-sample" } ], "indexes": [ { "index_statement": "CREATE INDEX adv_type ON `travel-sample`(`type`)", "keyspace_alias": "travel-sample", "recommending_rule": "Index keys follow order of predicate types: 2. equality/null/missing." } ] } } }, "query": "SELECT name, META().cas\nFROM `travel-sample`\nWHERE type=\"hotel\"\nORDER BY META().cas DESC\nLIMIT 2;" } ] }</span> |
2. Advise to adjust the order of index keys for functional and array predicates
This defect slightly changed the rules we had before:
- Delete the rule of functional index keys.
- Add rule “like” in the same position for scenario ” a like “%adv%”.This rule is applicable only when the pattern is of the type “%x”.If it is of the type “x%” the rule does not apply.
- Index Keys from functional predicates follow the order of EQ/in/LE/LT
A query like:
|
1 |
advise select * from shellTest where length(a21) <= 3 and c21 = 3 and c22 > 30 |
used to give
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
<span style="color: #ff0000">"results": [ { "#operator": "Advise", "advice": { "#operator": "IndexAdvice", "adviseinfo": { "recommended_indexes": { "indexes": [ { "index_statement": "CREATE INDEX adv_c21_c22_length_a21 ON `shellTest`(`c21`,`c22`,length((`a21`)))", "keyspace_alias": "shellTest", "recommending_rule": "Index keys follow order of predicate types: 2. equality/null/missing, 5. less than/between/greater than, 9. function index." } ] } } }, "query": "update shellTest set type=\"left\" where length(a21) < 3 and c21 = 3 and c22 > 30</span> |
now gives
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
<span style="color: #339966">{ "results": [ { "#operator": "Advise", "advice": { "#operator": "IndexAdvice", "adviseinfo": { "recommended_indexes": { "indexes": [ { "index_statement": "CREATE INDEX adv_c21_length_a21_c22 ON `shellTest`(`c21`,length((`a21`)),`c22`)", "keyspace_alias": "shellTest", "recommending_rule": "Index keys follow order of predicate types: 2. equality/null/missing, 4. not less than/between/not greater than, 5. less than/between/greater than." } ] }</span> <span style="color: #339966">} }, "query": "update shellTest set type=\"left\" where length(a21) <= 3 and c21 = 3 and c22 > 30" } ] } ]</span> |
3. Adjust the order of array index keys on its SATISFIES condition
- Put array predicates in the order of SATISFIES condition.
- For disjunction in SATISFIES: get the common terms -> pick the lowest priority one -> (EQ<IN<LE<LT<IS NOT NULL< LIKE)
- For conjunction in SATISFIES: pick the lowest priority one.
The new rules after fix#2 and #3:
1: leading array index for unnest,
2: equality/null/missing,
3: in,
4: not less than/between/not greater than,
5: less than/greater than,
6: derived join filter as leading key,
7: not null/not missing/valued,
8: like,
9: non-static join predicate,
10:flavor for partial index,
|
1 2 3 4 |
advise SELECT META(p).id, ARRAY_DISTINCT(IFMISSING(rooms[*].num,[])) FROM shellTest AS p WHERE (guestCode = IFNULL($guestCode, '') OR guestCode = '') AND checkinTime BETWEEN $startTime AND $endTime |
used to give
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
<span style="color: #ff0000">"results": [ { "#operator": "Advise", "advice": { "#operator": "IndexAdvice", "adviseinfo": { "recommended_indexes": { "covering_indexes": [ { "index_statement": "CREATE INDEX adv_guestCode_checkinTime_DISTINCT_rooms_level_size_num_array_di1747996179 ON `shellTest`(`guestCode`,`checkinTime`,DISTINCT ARRAY [`s`.`level`, `s`.`size`, `s`.`num`] FOR s in `rooms` END,array_distinct(ifmissing((array_star((`rooms`)).`num`), [])))", "keyspace_alias": "shellTest_p" } ], "indexes": [ { "index_statement": "CREATE INDEX adv_guestCode_checkinTime_DISTINCT_rooms_level_size_num ON `shellTest`(`guestCode`,`checkinTime`,DISTINCT ARRAY [`s`.`level`, `s`.`size`, `s`.`num`] FOR s in `rooms` END)", "keyspace_alias": "shellTest_p", "recommending_rule": "Index keys follow order of predicate types: 1. Common leading key for disjunction (2. equality/null/missing, 4. not less than/between/not greater than, 6. array predicate)." } ] } } }, "query": "SELECT META(p).id, ARRAY_DISTINCT(IFMISSING(rooms[*].num,[])) FROM shellTest AS p WHERE (guestCode = IFNULL($guestCode, '') OR guestCode = '') AND (checkinTime BETWEEN $startTime AND $endTime) AND (ANY s IN rooms SATISFIES [s.level,s.size, s.num] = [$level, $size, $num] END)" </span> |
now gives:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
<span style="color: #339966">results": [ { "#operator": "Advise", "advice": { "#operator": "IndexAdvice", "adviseinfo": { "recommended_indexes": { "covering_indexes": [ { "index_statement": "CREATE INDEX adv_guestCode_checkinTime_array_distinct_ifmissing_array_star_rooms_num ON `shellTest`(`guestCode`,`checkinTime`,array_distinct(ifmissing((array_star((`rooms`)).`num`), [])))", "keyspace_alias": "shellTest_p" } ], "indexes": [ { "index_statement": "CREATE INDEX adv_guestCode_checkinTime ON `shellTest`(`guestCode`,`checkinTime`)", "keyspace_alias": "shellTest_p", "recommending_rule": "Index keys follow order of predicate types: 1. Common leading key for disjunction (2. equality/null/missing, 4. not less than/between/not greater than)." } ] } } }</span> |
Give it a try and any problems you see with Indexes recommended by Index Advisor Service, you can add them as a comment to this blog
More about the Index Advisor feature:
https://www.couchbase.com/blog/index-advisor-service/
https://www.couchbase.com/blog/index-advisor-service-for-couchbase-n1qlfeb-refresh/
https://www.couchbase.com/blog/n1ql-index-advisor-improve-query-performance-and-productivity/
https://www.couchbase.com/blog/index-advisor-for-query-workload/
https://docs.couchbase.com/server/6.5/n1ql/n1ql-language-reference/advise.html
https://docs.couchbase.com/server/6.5/n1ql/n1ql-language-reference/advisor.html
https://docs.couchbase.com/server/6.5/tools/query-workbench.html#index-advisor
